Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds configurable submission rate limiting to the leaderboard system (stored per leaderboard GPU type in Postgres), exposes management/query surfaces via Discord commands and API endpoints, and standardizes several timestamps to UTC.
Changes:
- Add
rate_limit_secondstoleaderboard.gpu_typeand implement DB helpers to set/get/enforce limits. - Enforce rate limits during submission preparation and remove the prior hard-coded API-only rate limit.
- Add admin/user-facing commands/endpoints for rate limit configuration + various UTC timestamp updates / formatting changes.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
src/migrations/20260207_01_EW7ve-leaderboard-rate-limits.py |
DB migration adding rate_limit_seconds to leaderboard.gpu_type. |
src/libkernelbot/utils.py |
Makes parsed deadlines timezone-aware (UTC). |
src/libkernelbot/submission.py |
Calls DB to enforce submission rate limits before accepting a submission. |
src/libkernelbot/run_eval.py |
Records eval start/end timestamps in UTC. |
src/libkernelbot/leaderboard_db.py |
Implements rate limit storage and checks in the DB layer. |
src/libkernelbot/backend.py |
Uses UTC timestamps when creating submissions. |
src/kernelbot/cogs/verify_run_cog.py |
Updates verify-task leaderboard deadline to UTC (but still calls create_leaderboard incorrectly). |
src/kernelbot/cogs/misc_cog.py |
Adds a Discord command to view a leaderboard’s rate limits. |
src/kernelbot/cogs/admin_cog.py |
Adds an admin Discord command to set a leaderboard GPU rate limit. |
src/kernelbot/api/main.py |
Adds API endpoints to get/set rate limits; adjusts some timestamps/formatting. |
src/kernelbot/api/api_utils.py |
Removes prior hard-coded rate limit check in request validation. |
AGENTS.md |
Adds agent config pointer. |
.ruff.toml |
Increases Ruff line length to 120. |
Comments suppressed due to low confidence (1)
src/kernelbot/api/api_utils.py:145
_run_submissionconverts all exceptions into HTTP 400 responses. With rate limiting moved intoprepare_submission, this will also turn rate-limit errors into 400s. Consider catchingKernelBotErrorexplicitly and using itshttp_code(e.g., 429) when raisingHTTPException, while keeping other exceptions as 400/500 as appropriate.
async def _run_submission(submission: SubmissionRequest, mode: SubmissionMode, backend: KernelBackend):
try:
req = prepare_submission(submission, backend)
except Exception as e:
raise HTTPException(status_code=400, detail=str(e)) from e
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
Member
|
@claude review this code please |
msaroufim
approved these changes
Feb 8, 2026
Member
msaroufim
left a comment
msaroufim
left a comment
There was a problem hiding this comment.
We probably also want user based rate limits but LGTM
Member
|
wanna deal with merge conflicts and merge? we might need this as the competition is closing out soon |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.